It is well known that the dynamic range of an image captured with an image capture device (such as a photographic negative) is often greater than the dynamic range of the output medium (such as a photographic paper or CRT monitor). The result of this incongruity is that a good deal of scene content is lost in the output image. For this reason, in an image processing environment, a tone scale function may be used to reduce the scene dynamic range in order to map more information onto the output medium, in a process called dynamic range modification or dynamic range compression. The dynamic range compression modifies the tone scale characteristics of the image.
There exist many processes for creating a tone scale function on an image dependent basis; e.g. see U.S. Pat. No. 5,471,987 issued Dec. 5, 1995 to Nakazawa et al. Each of the conventional tone scale function processes examines certain statistical characteristics of the image under consideration in order to automatically generate the tone scale function. In addition, the tone scale function may be generated with manual interactive tools by a human operator.
After the tone scale function has been generated, there exists the question of how to apply the tone scale function to the digital image. The goal of dynamic range compression is to adjust the overall dynamic range of the image, rather than to affect the contrast of any given object in the image. In essence, a tone scale function should be applied to an image in such a way as to minimize the effect to the scene texture. To that end, it is known to apply a tone scale function to a low frequency sub-band of the image, preserving the higher frequency sub-band(s) that are considered image texture; e.g. see U.S. Pat. No. 5,012,333 issued Apr. 30, 1991 to Lee et al.
Lee et al. describe a procedure for preserving the high frequency detail of an image by blurring the image neutral channel in order to create a low-pass signal. Subtracting the low-pass signal from the image neutral channel produces a high-pass signal. The processed image is generated by applying the tone scale function to the low-pass signal and adding the result to the high-pass signal. This procedure preserves a segment of the image frequency spectrum; however, artifacts are generated at object boundaries in the image. Gallagher et al. build on this work; see U.S. Pat. No. 6,317,521 issued Nov. 13, 2001. More specifically, Gallagher et al. incorporate an artifact avoidance scheme along with a single standard FIR filter to generate the texture signal. While this improvement reduces the occurrence of artifacts in the final image, the artifacts can still be visible.
Several methods for achieving dynamic range modification of an image by decomposing the image into multiple resolutions have been proposed. For example, in U.S. Pat. No. 5,467,404 issued Nov. 14, 1995, and U.S. Pat. No. 5,805,721 issued Sep. 8, 1988, Vuylsteke et al. teach a method of decomposing an image into a pyramid having multiple resolution versions of the image and using a pre-determined nonlinear amplitude compression function for the high frequency component in each resolution. A deficiency of this method is that the amplitude at each resolution does not adequately identify whether the signal is part of a large amplitude edge or an image texture. A similar approach was disclosed in U.S. Pat. No. 5,717,791 issued Feb. 10, 1998 to Labaere et al., which describes a similar dynamic range compression scheme using an image pyramid generated using wavelet filters to generate the multiple resolutions.
In U.S. Pat. No. 5,907,642 issued May 25, 1999, Ito describes a method of image enhancement based on processing the detail signals of an image pyramid. Ito describes suppressing the magnitude of detail signals in situations where the next lower detail signal has small magnitude. In U.S. Pat. No. 5,991,457 issued Nov. 23, 1999, Ito describes a method of generating several band pass detail image signals that are modified by application of non-linear functions to modify the dynamic range of the image.
In U.S. Pat. No. 6,285,798 B1 issued Sep. 4, 2001, Lee describes yet another dynamic range compression method using a pyramid representation of an image. Lee describes a method of using wavelet filters to create a plurality of coarse signals and detail signals, modifying the detail signals in accordance with contrast gain signals created by detecting the edges of the coarse scale edges, and adding the modified detail signals to the coarse signals to obtain an output image.
In each of these dynamic range compression techniques using a pyramid image representation, the high frequency (e.g. edge or band pass) components of the pyramid image representation are modified to affect the image dynamic range. However, it is often inconvenient to operate on the high frequency component of the pyramid image representation. In addition, the characteristics of the high frequency signals vary as a function of the level within the pyramid image representation. This variability requires a complicated parameter tuning in order to achieve optimal dynamic range compression without producing objectionable artifacts (such as the aforementioned overshoot and undershoot artifact) using a pyramid representation of the image.
Pyramid methods as a means of representing images as a function of spatial resolution for image processing, has a long history. Burt and Adelson, described a method of representing a digital image by a series of residual images and a base digital image in their journal article “The Laplacian Pyramid as a Compact Image Code” IEEE Transactions on Communications, Vol. Com-31, No. 4, Apr. 1983. However the method taught by Burt et al. was designed for image compression applications and cannot be used for enhancing the tone scale of a digital image.
The prior art methods described for handing the borders of an image pyramid are inefficient. Vuylsteke, LaBaere, and Ito each describe generating lower resolution base image pyramid levels by filtering then sampling every other pixel. This method throws away image information whenever the image has an even number of rows, as the samples of the lower resolution base image span (cover) all but one of the image rows. In addition, the sampling method does not preserve phase between the resolution levels. For example, consider an 8×8 pixel image. Omitting the blurring filter, after 4 pyramid levels have been generated, the image is represented as a single pixel. The value of the single pixel is the same as the pixel of the 8×8 image in the upper left position (assuming sampling of every other pixel.) Thus a phase shift of several pixels has occurred. Because of this phase shift, the pyramids used by Vuylsteke, Labaere and Ito are sub-optimal.
In addition, Gendel describes an image pyramid in U.S. Pat. No. 6,141,459, issued Oct. 31, 2000. Gendel describes a method of forming an image pyramid to handle the image borders so that a smoothing filter will provide valid results. However, Gendel's solution of padding the image is inefficient and the method does not ensure that image information is not lost due to the span of the lower resolution base image being smaller than the starting base image.
Therefore, there exists a need for an improved method of forming image pyramids and in particular an improved method for processing image pyramid borders. Specifically, there is a need to improve the method of processing image pyramid borders when using an image pyramid to modify the tone scale of a digital image.